Image capturing device, image capturing method, and information distribution system

ABSTRACT

An image capturing device includes: a display section attached to a head or a face of a user, and configured to display an image; a communication section configured to communicate with an external device; an image capturing section; a positional information acquisition section configured to acquire current positional information of the user; and a control section configured to capture an image of a dangerous place by the image capturing section in accordance with determination of a dangerous state, and transmit a captured image from the communication section to the external device together with the positional information acquired by the positional information acquisition section.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 14/458,792 filed Aug. 13, 2014 which claims thebenefit of Japanese Priority Patent Application JP 2013-173398 filedAug. 23, 2013, the entire contents of which are incorporated herein byreference.

BACKGROUND

The present disclosure relates to an image capturing device and an imagecapturing method that provide images regarding a dangerous placeincluding a source of an accident or an incident, and an informationdistribution system that distributes information regarding a dangerousplace obtained based on images provided from one or more image capturingdevices.

The knowledge of a system for sharing information obtained by monitoringautomobiles while driving among vehicles is widespread. The sharedinformation is sometimes referred to as driving information, navigationinformation, or vehicle information, for example. For example, aproposal has been made of an image collection system that automaticallycollects images in the vicinity of an accident occurrence site (forexample, refer to Japanese Unexamined Patent Application Publication No.2012-98105). The image collection system includes: a plurality ofvehicles including a car navigation system, a impact detection sectionthat detects the occurrence of an accident, an in-vehicle camera, and astorage section that stores an image captured by the in-vehicle camerain association with the corresponding time, and an informationmanagement center. When information regarding an accident is transmittedfrom a certain collision-damaged vehicle to the information managementcenter, the information management center notifies the occurrence of theaccident to the other vehicles, and obtains images captured by thein-vehicle cameras of the other vehicles.

In the above-described image collection system, images at a time wheneach traffic accident occurred are collected from vehicles other than acollision-damaged vehicle, and thus analysis precision of the cause ofthe accident is intended to be increased on the basis of the images froma plurality of viewpoints.

However, the images collected by the above-described image collectionsystem are limited to the images captured by in-vehicle cameras. That isto say, the collected images are all limited to the viewpoints of theindividual in-vehicle cameras. Thus, those images are different from theimages from the viewpoints of the drivers, and pedestrians who happenedto be at an accident site. Accordingly, those images are not necessarilyimages that are obtained from easily viewable and suitable viewpoints.

Also, the above-described image collection system basically startsprocessing in response to the detection of the occurrence of anaccident, and collects images of a site where an accident has actuallyhappened.

SUMMARY

It is desirable to provide an excellent image capturing device and animage capturing method that are capable of suitably obtaining andproviding images regarding a dangerous place including a source of anaccident or an incident.

It is also desirable to provide an excellent information distributionsystem capable of suitably distributing information regarding adangerous place on the basis of images of the dangerous place, whichwere provided from image capturing devices.

According to an embodiment of the present disclosure, there is providedan image capturing device including: a display section attached to ahead or a face of a user, and configured to display an image; acommunication section configured to communicate with an external device;an image capturing section; a positional information acquisition sectionconfigured to acquire current positional information of the user; and acontrol section configured to capture an image of a dangerous place bythe image capturing section in accordance with determination of adangerous state, and transmit a captured image from the communicationsection to the external device together with the positional informationacquired by the positional information acquisition section.

The image capturing device according to the above-described embodimentmay further include at least either a state information acquisitionsection configured to acquire state information of the user, or anenvironmental information acquisition section configured to acquireenvironmental information, wherein the control section may determine thedangerous state on the basis of the state information or theenvironmental information.

In the above-described embodiment, the control section may transmit thecaptured image with at least either information on image captured timeor attribute information of the user.

In the above-described embodiment, the state information acquisitionsection may acquire biological information of the user, and the controlsection may determine a dangerous state by itself on the basis of thebiological information, or acquires a determination result of a relativedangerous state determined by an external apparatus recording a largenumber of pieces of data.

In the above-described embodiment, the control section may determine thedangerous state on the basis of at least any one piece of biologicalinformation among a pulse of the user, perspiration, myoelectricpotential, and movement of an eyeball.

In the above-described embodiment, the control section may determine thedangerous state further in consideration of whether the user is viewinga content using the display section.

In the above-described embodiment, the control section may determine thedangerous state on the basis of an analysis result of ambient noises,winds, atmosphere (a thick fog or smoke, radioactive rays, or toxicchemical substance in an atmosphere), surrounding captured images as theenvironmental information.

The image capturing device according to the above-described embodimentmay further include a line of sight acquisition section configured toacquire a line of sight of the user, wherein the control sectiondetermines the dangerous state on the basis of biological information ofthe user when the line of sight falls on a place predicted to bedangerous.

In the above-described embodiment, the control section may identify theplace predicted to be dangerous on the basis of the environmentalinformation.

The image capturing device according to the above-described embodimentmay further include a line of sight acquisition section configured toacquire a line of sight of the user, wherein when the control sectionmay determine the dangerous state, the control section captures a stillimage or a moving image by the image capturing section in accordancewith whether or not the line of sight remained for a certain timeperiod.

In the above-described embodiment, the control section may stopcapturing the image in accordance with a change in the line of sightwhile capturing the moving image.

The image capturing device according to the above-described embodimentmay further include a line of sight acquisition section configured toacquire a line of sight of the user, wherein the control section mayidentify the dangerous place, starts capturing a moving image inaccordance with the line of sight falling on the dangerous place, andstops capturing the moving image in accordance with the line of sightfalls off the dangerous place.

In the above-described embodiment, the control section may determinewhether to capture a still image or a moving image in accordance with alevel of importance of the dangerous place or a type of the dangerousplace.

According to another embodiment of the present disclosure, there isprovided a method of acquiring information, including: determining adangerous state on the basis of state information or environmentalinformation of a user; acquiring positional information in order toacquire current positional information of the user; capturing an imageof a dangerous place by an image capturing section in accordance withdetermination of the dangerous state; and transmitting a captured imagefrom a communication section to an external device together with thepositional information acquired by the acquiring positional information.

According to another embodiment of the present disclosure, there isprovided an information distribution system including: an image databaseconfigured to manage an image of a dangerous place together withpositional information; and an information providing section configuredto retrieve an image of a dangerous place corresponding to a currentposition of an information display device from the image database inorder to provide the image.

Note, however, that a “system” mentioned here means a logical set of aplurality of devices (or functional modules that achieve specificfunctions), and thus it does not matter whether the individual devicesor the functional modules are contained in a single case or not.

In the information distribution system according to the embodiment, theimages of the dangerous place collected from one or more imagescapturing devices may be managed with individual pieces of positionalinformation, respectively, in the image database.

In the information distribution system according to the embodiment, timeinformation when the image of the dangerous place was captured by theimage capturing device may be managed in the image database incombination, and the information providing section may retrieve an imageof the dangerous place corresponding to the current position of theinformation display device from the image database by filtering with acurrent time.

The information distribution system according to the above-describedembodiment may further include a user database configured to manageattribute information of a user of each image capturing device, whereinthe information providing section retrieves an image of the dangerousplace corresponding to the current position of the information displaydevice from the image database by filtering with attribute informationof a user of an information display device of the providing destinationin order to provide the image.

In the information distribution system according to the embodiment, theinformation providing section may provide a hazard map created bymapping each dangerous place stored in the image database onto a map.

In the information distribution system according to the embodiment, whena plurality of images are retrieved for the current position of theinformation display device, the information providing section may narrowdown the images to a predetermined number of images using any one of thefollowing methods, (A) performing an image search using a default searchcondition, (B) performing an image search using a search conditionspecified by an information display device, and (C) transmittingnecessary information to an information display device, and theinformation display device performing an image search using a desiredsearch condition.

With a technique disclosed in this specification, it is possible toprovide an excellent image capturing device and an image capturingmethod that are capable of suitably obtaining and providing imagesregarding a dangerous place including a source of an accident or anincident.

Also, with a technique disclosed in this specification, it is possibleto provide an excellent information distribution system capable ofsuitably distributing information on a dangerous place on the basis ofimages regarding the dangerous place, which were provided from imagecapturing devices.

In this regard, the advantages described in this specification are onlyexamples. The advantages of this disclosure are not limited to this.Also, additional advantages may further be given by the presentdisclosure in addition to the above-described advantages.

Other objects, features, and advantages of the technique disclosed inthis specification will become apparent from the more detaileddescription of the embodiments described below with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically illustrating a configuration of aninformation distribution system according to an embodiment of atechnique disclosed in this specification;

FIG. 2 is a front view of a state of a user wearing an image capturingdevice configured as a transmissive head mounted display;

FIG. 3 is a top view of the user wearing the image capturing deviceillustrated in FIG. 2;

FIG. 4 is a front view of a state of a user wearing an image capturingdevice configured as a shading head mounted display;

FIG. 5 is a top view of the user wearing the image capturing deviceillustrated in FIG. 4;

FIG. 6 is a diagram illustrating an example of an internal configurationof the image capturing device;

FIG. 7 is a diagram schematically illustrating a functionalconfiguration of an information management center;

FIG. 8A is a diagram illustrating an example of a data structure formanaging images in an image database;

FIG. 8B is a diagram illustrating an example of a data structure formanaging images in an image database;

FIG. 8C is a diagram illustrating an example of a data structure formanaging images in an image database;

FIG. 9 is a diagram illustrating an example of a data structure formanaging users in a user database;

FIG. 10 is a diagram schematically illustrating a functionalconfiguration of an information display device;

FIG. 11 is a diagram schematically illustrating an example of aconfiguration of an actuator section;

FIG. 12 is a flowchart illustrating a general processing procedure forcapturing an image of a dangerous place, which is executed in each ofthe image capturing devices;

FIG. 13 is a flowchart illustrating determination processing of adangerous state, which is executed by the image capturing device in stepS1201;

FIG. 14 is a flowchart illustrating a variation of the determinationprocessing of a dangerous state, which is executed by the imagecapturing device in step S1201;

FIG. 15 is a flowchart illustrating another variation of thedetermination processing of a dangerous state, which is executed by theimage capturing device in step S1201;

FIG. 16 is a flowchart illustrating a processing procedure for capturingan image of a dangerous place, which is executed by the image capturingdevice in step S1203;

FIG. 17 is a flowchart illustrating another processing procedure forcapturing an image of a dangerous place, which is executed by the imagecapturing device in step S1203;

FIG. 18 is a flowchart illustrating still another processing procedurefor capturing an image of a dangerous place, which is executed by theimage capturing device in step S1203;

FIG. 19 is a flowchart illustrating still another processing procedurefor capturing an image of a dangerous place, which is executed by theimage capturing device in step S1203;

FIG. 20 is a flowchart illustrating a procedure for the informationmanagement center to process an image of a dangerous place received fromthe image capturing device;

FIG. 21 is a flowchart illustrating a processing procedure for theinformation management center to transmit an image of a dangerous placeor information thereof to the information display device;

FIG. 22 is a flowchart illustrating a processing procedure for theinformation display device to display information regarding a dangerousplace, received from the information management center, to a user;

FIG. 23 is a flowchart illustrating a processing procedure fordetermining a dangerous state using behavior characteristic andphysiological phenomena of a person;

FIG. 24 is a diagram illustrating a display example of an image of adangerous place on the information display device;

FIG. 25 is a diagram illustrating a display example of an image of adangerous place on the information display device;

FIG. 26 is a diagram illustrating a display example of an image of adangerous place on the information display device; and

FIG. 27 is a diagram illustrating a display example of an image of adangerous place on the information display device.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following, a detailed description will be given of an embodimentof a technique disclosed in this specification with reference to thedrawings.

FIG. 1 schematically illustrates a configuration of an informationdistribution system 100 according to an embodiment of a techniquedisclosed in this specification. The information distribution system 100illustrated in FIG. 1 includes image capturing devices 110-1, 110-2,110-3, . . . , which capture and provide images of a dangerous placeincluding a source of an accident and an incident, an informationmanagement center 120, which collects images provided from theindividual image capturing devices 110-1, 110-2, 110-3, . . . , anddistributes information on the dangerous place, and information displaydevices 130-1, 130-2, 130-3, . . . , which receive the informationdistributed from the information management center 120 to display theinformation.

In this regard, it is assumed that the information management center 120is a server apparatus built on a wide area network, such as theInternet, and the like. However, the information management center 120may be replaced by services provided using cloud technology, or some ofthe image capturing devices 110 or the information display devices 130may share the functions of the information management center 120. Also,it is assumed that information transfer is performed between the imagecapturing device 110 and the information display device 130 by apoint-to-point connection without communication through the informationmanagement center 120.

In the present embodiment, each of the image capturing devices 110-1,110-2, 110-3, . . . , is basically configured in a style in which a userwears the image capturing device on his or her head or face in themanner of a head mounted display, and is used all the time when the useris walking or driving a vehicle, or the like. A description will begiven later of a specific configuration of the head mounted display.

On the other hand, the information display devices 130-1, 130-2, 130-3,. . . , display information to moving users. Of course, it is possibleto display information to a user who remains indoors. The informationdisplay devices 130-1, 130-2, 130-3, . . . , may be a multifunctionalmobile terminal, for example, a head mounted display, a smart phone, orthe like, or a head up display, or the like. Among these, a head updisplay is applied to a windshield of a vehicle, or the like, and forexample, is capable of displaying an enlarged information to a visualfield of a user who is in the vehicle. Also, some of the individualimage capturing devices 110-1, 110-2, 110-3, . . . may function as theinformation display device 130 in a situation other than the case ofcollecting images.

When a user who is wearing each of the image capturing devices 110-1,110-2, 110-3, . . . , encounters a danger, a corresponding imagecapturing device transmits an image of the site (a moving image or astill image) to the information management center 120 together with thepositional information of the site. At that time, each image capturingdevice may transmit attached information, such as time of the encounterwith the danger, information regarding an encountered danger category(an accident, an incident, or a minor incident) (or an information tagto be used for identifying a danger category), user attributeinformation including a personal attribute of the user (gender, age,character, occupation, and the like), a behavioral attribute, and thelike. In the case where the information management center 120 managesuser attribute information in a database, each image capturing devicemay transmit only a user-ID.

Each of the image capturing devices 110-1, 110-2, 110-3, . . . ,automatically detects that the user who is wearing the device hasencountered a dangerous place on the basis of the user's biologicalinformation, and the like, and automatically captures an image of thesite. Of course, the user may notify the image capturing device that theuser has encountered a dangerous place by manual operation. A detaileddescription will be given later of a method of automatically detectingan encounter with a dangerous place.

The information management center 120 manages images of the dangerousplace that have been transmitted, that is to say, provided from each ofthe image capturing devices 110-1, 110-2, 110-3, . . . , in associationwith positional information. Also, the information management center 120may classify the images provided from each of the image capturingdevices 110-1, 110-2, 110-3, . . . , for each danger category asnecessary, and may manage the images in association not only with thepositional information, but also with the other information, such as theuser attribute information (described above), and the like.

The information management center 120 monitors information on thecurrent position of each of the information display devices 130-1,130-2, 130-3, . . . , all the time. Alternatively, the informationdisplay device 130 that requests information regarding a dangerous placemay transmit the information on the current position of itself to theinformation management center 120.

Then, the information management center 120 makes a search with acurrent position of each of the information display devices 130-1,130-2, 130-3, . . . . Then, if a matching dangerous place is found, theinformation management center 120 transmit a captured image of the site,or information regarding a dangerous place obtained by an image analysisto a corresponding information display device 130.

Also, the information management center 120 may perform matching notonly using the current position, but may perform filtering with apersonal attribute or a behavioral attribute of a user who uses each ofthe information display devices 130-1, 130-2, 130-3, . . . and then maytransmit an image or information regarding a dangerous place obtained byimage analysis if only necessary. This is because there are sites thatare dangerous to women, and but not to men, sites that are dangerous toinfants or elderly people in particular, and there are cases where it isdangerous when running, but not dangerous in particular and unnecessaryto pay attention when walking, and the like. If information istransmitted to a user who is not in danger, it becomes a uselesswarning, and thus information will not be accepted seriously in the caseof a real danger.

When each of the information display devices 130-1, 130-2, 130-3, . . ., receives an image of a dangerous place, or information regarding adangerous place obtained by image analysis from the informationmanagement center 120, the information display device displays the imageor the information so as to give a warning against a danger to the userwho is carrying the device.

For example, if the information display device 130 is a head mounteddisplay, or a head up display, information is displayed by beingoverlaid on a dangerous place in the visual range of the user so as toeffectively notify the user of a danger in order to avoid the danger.

Also, the information display device 130 may further call user'sattention by giving sound output, such as sound effect, or the like,vibration, performing tactile feedback in addition at a time whendisplaying a captured image of a dangerous place.

As described above, each of the image capturing devices 110-1, 110-2,110-3, . . . , is basically configured in the form of a head mounteddisplay to be used by the user wearing the device on his/her head orface, and is used by the user wearing the device all the time while theuser is walking or driving a vehicle.

FIG. 2 illustrates a front view of a state of a user wearing the imagecapturing device 110 configured as a transmissive head mounted display.The image capturing device 110 illustrated in FIG. 2 is transmissive,that is to say, see-through type, and thus it is possible for the userto view a scene of a real world (that is to say, see-through) throughthe image while an image is displayed to the user. Accordingly, it ispossible to view a virtual display image, such as an AR (AugmentedReality) image, and the like on a scene of a real world (for example,refer to Japanese Unexamined Patent Application Publication No.2011-2753). Also, the displayed image is difficult to see from theoutside (that is to say, by other people), and thus privacy is easy tobe protected when information is displayed.

The image capturing device 110 illustrated in FIG. 2 has a structuresimilar to glasses for vision correction. Virtual image optical sections201L and 201R including transparent light guiding sections, and thelike, respectively, are disposed at positions opposed to right and lefteyes of a user of the main body of the image capturing device 110. Animage observed by the user (not illustrated in FIG. 2) is displayedinside each of the virtual image optical sections 201L and 201R. Each ofthe virtual image optical sections 201L and 201R is supported by aholding body 202, which is like a frame of glasses, for example.

An outside camera 612 for inputting a surrounding image (user's visualrange) is disposed substantially at the center of the holding body 202glasses, which is like a frame of glasses. The outside camera 612 iscapable of capturing a scene in the direction of the line of sight ofthe user, for example. It is desirable to configure the outside camera612 by a plurality of cameras in order to obtain three-dimensionalinformation of a surrounding image using binocular disparityinformation.

Also, microphones 203L and 203R are disposed in the vicinity of both ofthe right and left ends of the holding body 202, respectively. Bysubstantially holding the microphones 203L and 203R in bilaterallysymmetrical manner, it is possible to recognize only sound that isoriented in the center (user' voice). Thereby, it is possible toseparate the user' voice from ambient noises, and speech sound of theother people, and thus to prevent malfunction at the time of operationby voice input, for example.

Also, outside display sections 615 which enables external people toobserve outside images are disposed at the outside of the main body ofthe image capturing device 110. In the example in FIG. 2, a pair ofright and left outside display sections 615 are disposed. However, asingle or not less than three display sections 615 may be disposed. Theoutside images may be either the same image as that of a display section609, or a different image.

FIG. 3 illustrates a top view of the user wearing the image capturingdevice 110.

As illustrated in FIG. 3, display panels 204L and 204R for outputtingleft-eye and right-eye images are disposed at both of the left and rightends of the image capturing device 110, respectively. Each of thedisplay panels 204L and 204R includes a micro display, such as a liquidcrystal display or an organic EL element, or the like, or a laserscanning method display, such as a retinal direct drawing display, orthe like. The left and right display images output from the displaypanels 204L and 204R are led in the vicinity of left and right eyesthrough the virtual image optical sections 201L and 201R, respectively,and the enlarged virtual image is formed on the pupil of a user.Although the detailed illustration is omitted, the virtual image opticalsections 201L and 201R include an optical system condensing lightemitted from the micro display, an optical waveguide disposed at theposition of entering light passing through the optical system, apolarization filter reflecting incident light to the optical waveguide,and a polarization filter outputting light propagated by being totallyreflected in the optical waveguide to the eye of the user, respectively.

Also, FIG. 4 illustrates a front view of a state of a user wearing theimage capturing device 110 configured as a shading head mounted display.

The image capturing device 110 directly covers the eyes of the user whenthe user wears the device on his/her head or face, and thus allows togive the user who is viewing an image an immersion feeling. Also, unlikethe see-through type, the user who is wearing the image capturing device110 is not allowed to directly see a scene of a real world. However, byproviding the outside camera 612 that captures an image of a scene inthe user' direction of the line of sight, and displaying the capturedimage, it is possible for the user to indirectly view a scene of a realworld (that is to say, display a scene by video see-through). Ofcourser, it is possible to display a virtual display image, such as anAR image, in an overlaying manner on a video see-through image. Also,the display image is not allowed to be seen from the outside (that is tosay, by the other people), it is easy to protect privacy at the time ofinformation display.

The image capturing device 110 illustrated in FIG. 4 has a structurehaving a similar shape as a headgear, and the image capturing device 110directly covers left and right eyes of the user who is wearing thedevice. The display panels observed by the user (not illustrated in FIG.4) are disposed at positions opposed to right and left eyes of a user ofthe inner main body of the image capturing device 110. The display panelincludes a micro display, such as an organic EL element or a liquidcrystal display, or the like, for example, or a laser scanning methoddisplay, such as a retinal direct drawing display, or the like.

An outside camera 612 for inputting a surrounding image (user's visualrange) is disposed substantially at the center of the front body of theimage capturing device 110. Also, microphones 403L and 403R are disposedin the vicinity of both of the left and right ends of the main body ofthe image capturing device 110, respectively. By substantially holdingthe microphones 403L and 403R in bilaterally symmetrical manner, it ispossible to recognize only sound that is oriented in the center (user'voice). Thereby, it is possible to separate the user' voice from ambientnoises, and speech sound of the other people, and to thus preventmalfunction at the time of operation by voice input, for example.

FIG. 5 illustrates a top view of the user wearing the image capturingdevice 110 illustrated in FIG. 4. The image capturing device 110illustrated in FIG. 5 includes the display panels 404L and 404R for aleft-eye and a right-eye, respectively, at the side face of the opposedside to the user's face. Each of the display panels 404L and 404Rincludes a micro display, such as an organic EL element, or a liquidcrystal display, or the like, for example, or a laser scanning methoddisplay, such as a retinal direct drawing display, or the like. Thedisplay images of the display panels 404L and 404R are observed by theuser as an enlarged virtual image by passing through the virtual imageoptical sections 401L and 401R, respectively. Also, an eye height and aneye width are different for each user, and thus it is necessary toposition each of the left and right display systems and the user's eyes,respectively. In the example in FIG. 5, an eye-width adjustmentmechanism 405 is provided between the right-eye display panel and theleft-eye display panel.

Also, outside display sections 615 which enables external people toobserve outside images are disposed at the outside of the main body ofthe image capturing device 110. In the example in FIG. 5, a pair ofright and left outside display sections 615 are disposed. However, asingle or not less than three display sections 615 may be disposed. Theoutside images may be either the same image as that of a display section609, or a different image.

FIG. 6 illustrates an example of an internal configuration of the imagecapturing device 110. Both of the transmissive type illustrated in FIG.2 and FIG. 3, and the immersion type illustrated in FIG. 4 and FIG. 5have substantially the same configuration. In the following, adescription will be given of each section.

A control section 601 includes a ROM (Read Only Memory) 601A and a RAM(Random Access Memory) 601B. The ROM 601A stores the program code to beexecuted in the control section 601, and various kinds of data. Thecontrol section 601 executes the program loaded in the RAM 601B so as tototally control overall operation of the image capturing device 110 as ahead mounted display including display control of an image. The programsand the data that are stored in the ROM 601A include an image displaycontrol program, an image providing program that detects an encounterwith a dangerous place by the device 110 (or by the user wearing thecorresponding device 110), and captures an image of the dangerous placeto transmit the image to the information management center 120, specificidentification information of the device 110, user attribute information(described above) of the user who uses the device 110, and the like.

An input operation section 602 includes one or more operators with whichthe user performs input operation, such as a key, a button, a switch,and the like, and receives a user' instruction through the operators,and outputs the instruction to the control section 601. Also, the inputoperation section 602 accepts a user's instruction included in a remotecontrol command received by the remote control reception section 603,and outputs the instruction to the control section 601.

A state information acquisition section 604 is a functional module forobtaining state information of the main body of the image capturingdevice 110, or a user who is wearing the device 110. The stateinformation acquisition section 604 may includes various sensors fordetecting the state information of itself, or may obtain the stateinformation from an external device (for example, a watch or a smartphone worn by the user, and the other multifunctional terminals)including a part of or all of these sensors through a communicationsection 605 (described later).

The state information acquisition section 604 obtains information on theposition and posture of a user' head, and posture, for example. Thestate information acquisition section 604 may include any one of or acombination of two or more sensors in consideration of merits anddemerits of each sensor of a gyro sensor, an acceleration sensor, a GPS(Global Positioning System) sensor, a geomagnetic sensor, a Dopplersensor, an infrared sensor, a radio field intensity sensor, and the likein order to obtain position and posture information. Also, the stateinformation acquisition section 604 may use a combination of informationprovided from various kinds of infrastructure, such as mobile-phone basestation information, PlaceEngine (registered trademark) information(radio measurement information from a wireless LAN access point), andthe like in order to obtain the position and posture information.

Also, a state information acquisition section 504 obtains, for example,the user' working state (whether the user is wearing the image capturingdevice 110 or not), the user's behavioral state (movement state, such asstationary, walking, running, or the like, an open or shut state of aneyelid, a direction of line of sight, the size of a pupil), a mentalcondition (a degree of impression, such as whether the user is immersedor concentrated while observing a display image, and the like, a degreeof excitement, a degree of arousal, feeling, emotion, and the like), andfurther, a physiological state as state information of the user who iswearing the image capturing device 110. Also, the state informationacquisition section 604 may include various state sensors, such as awearable sensors including a mechanical switch, and the like, an innercamera to capture a user's face, a gyro sensor, an acceleration sensor,a speed sensor, a pressure sensor, a temperature sensor to detect a bodytemperature or air temperature, a perspiration sensor, a pulse sensor, amyoelectric potential sensor, an electro oculography sensor, a brainwavesensor, a breath sensor, a gas ion concentration sensor, and a timer(all of these are not illustrated in FIG. 6) in order to obtain thesepieces of state information from the user.

An environmental information acquisition section 616 is a functionalmodule that obtains information on the main body of the image capturingdevice 110, or an environment surrounding the user who is wearing thedevice 110. The information regarding environment here includes sound,air flow, air temperature, atmospheric pressure, atmosphere (smoke,thick fog, electromagnetic waves to which the device 110 or the user isexposed (ultraviolet ray, blue light, and radio wave), heat ray(infrared ray), radioactive ray, carbon monoxide and carbon dioxide inthe atmosphere, nitrogen compound (nicotine), nitrogen oxides (NOx)floating in the atmosphere, and a hydrocarbon (volatile organiccompounds (VOC) or, photochemical smog produced by photochemicalreaction under the influence of ultraviolet rays on these, powder dust,such as particulate matter, pollen, house dust, and the like, toxicchemicals, such as asbestos), and the other environmental factors. Theenvironmental information acquisition section 616 may include variousenvironmental sensors including a sound sensor, air flow sensor in orderto detect environmental information. The above-described microphone andthe outside camera 612 may be included in the environmental sensor.Alternatively, the environmental information acquisition section 616 mayobtain the environmental information from an external device including apart of or all of these sensors (for example, a smart phone, a watch,and the other multifunctional terminals that are worn by the user)through the communication section 605 (described later).

The outside camera 612 is disposed substantially at the center of thefront face of the main body of the image capturing device 110 (refer toFIG. 2 and FIG. 4), for example, and is capable of capturing thesurrounding image. Also, by performing posture control of the outsidecamera 612 in a panning, a tilting, and a rolling direction insynchronism with the direction of the user's line of sight obtained bythe state information acquisition section 604, it is possible to capturean image in the user's eye direction, that is to say, an image in thedirection of the user's line of sight, by the outside camera 612. It isfurther preferable to configure the outside camera 612 by a plurality ofcameras using binocular disparity information in order to obtain threedimensional information of the surrounding image. It is assumed that theuser is allowed to adjust zooming of the outside camera 612 throughoperation of the input operation section 602, the size of the pupil ofthe eye, which is recognized by an inside camera, a myoelectricpotential sensor, or the like, and voice input. It is possible to outputthe image captured by the outside camera 612 to the display section 609,and also, to store the image into a storage section 606. In the presentembodiment, an image of a dangerous place that the user has encounteredis captured by the outside camera 612. The details thereof will bedescribed later.

The communication section 605 performs communication processing with anexternal device, and modulation/demodulation, and encoding/decodingprocessing on a communication signal. As an example of the externaldevice, it is possible to give a content playback apparatus (a Blu-raydisc or a DVD player), which provides a viewing content at a time whenthe user uses the image capturing device 110 as a head mounted display,a streaming server, and an information management center 120 on theInternet. Also, the control section 601 outputs transmission data to anexternal device from the communication section 605.

The communication section 605 may have any configuration. For example,it is possible to configure the communication section 605 in accordancewith a communication method to be used for transmission/receptionoperation with an external device to be the other party of thecommunication. The communication method may be either wired or wirelessmode. The communication standards mentioned here include MHL (MobileHigh-definition Link), USB (Universal Serial Bus), HDMI (registeredtrademark) (High Definition Multimedia Interface), Wi-Fi (registeredtrademark), Bluetooth (registered trademark) communication, BLE(Bluetooth (registered trademark) Low Energy) communication,ultralow-power wireless communication, such as ANT and the like, a meshnetwork standardized by IEEE802.11s, and the like. Alternatively, thecommunication section 605 may be a cellular radio transmitter/receiver,which operates in accordance with standard specifications, such asW-CDMA (Wideband Code Division Multiple Access), LTE (Long TermEvolution), and the like, for example.

As described later, an image of a dangerous place, or the like, which iscaptured by the outside camera 612, is transmitted from thecommunication section 605 to the information management center 120.However, the captured image itself is sometimes not transmitted to theinformation management center 120, but is uploaded to an image sharingsite, and a URL (Uniform Resource Locator), or a URI (Uniform ResourceIdentifier), which indicates information for accessing an uploadedimage, may be transmitted to the information management center 120.

The storage section 606 is a mass storage device including an SSD (SolidState Drive), or the like. The storage section 606 stores applicationprograms to be executed by the control section 601, and various kinds ofdata. For example, a content viewed by the user using the imagecapturing device 110 as a head mounted display is stored in the storagesection 606. Also, in the present embodiment, a scene image captured bythe outside camera 612 (dangerous place, and the like) is stored in thestorage section 606 in association with the positional information andthe time information, which were obtained by the state informationacquisition section 604, and further, the environmental informationobtained by the environmental information acquisition section 616.

An image processing section 607 further performs signal processing, suchas image quality correction, and the like on the image signal outputfrom the control section 601, and performs resolution conversion to meetthe screen of the display section 609. Then, the display drive section608 selects the pixels of the display section 609 for each line insequence, performs line sequential scanning, and supplies a pixel signalbased on the image signal having been subjected to the signalprocessing.

The display section 609 includes a display panel including a microdisplay, such as a an organic EL (Electro-Luminescence) element, aliquid crystal display, or the like, for example, or a laser scanningmethod display, such as a retinal direct drawing display, or the like.The virtual image optical section 610 performs extended projection ofthe display image of the display section 609 to allow the user toobserve an enlarged virtual image.

In this regard, as an example of the display image output by the displaysection 609, it is possible to give a content playback apparatus (aBlu-ray disc or a DVD player), or a commercial content supplied from astreaming server, and a captured image by the outside camera 612, andthe like. If the image capturing device 110 is a see-through type asillustrated in FIG. 2 and FIG. 3, it is possible for the user to view ascene of a real world through the image (including a time when an imageis not displayed).

Also, the display screen of the outside display section 615 faces theoutside of the image capturing device 110 (in the opposite direction ofthe face of the user wearing the device), and thus it is possible todisplay the outside image to the other users. The outside image may beeither the same image as that of the display section 609, or a differentimage. For the detailed configuration of the outside display section615, refer to the specifications of Japanese Patent Application Nos.2012-200902 and 2012-200903, respectively, which have already beenassigned to the present applicant, for example. Note that the outsidedisplay section 615 is not an indispensable component for achieving thetechnique disclosed in this specification.

An audio processing section 613 performs sound quality correction andsound amplification on the audio signal output from the control section601, and further performs signal processing on the input audio signal,and the like. Then, an audio input/output section 614 outputs the soundafter having been subjected to the audio processing, and performs audioinput from the microphone (described above).

The information management center 120 is a server apparatus, or the likeon a wide area network, such as the Internet, and can be built using ageneral computer system. Alternatively, the information managementcenter 120 may not be physically configured in a single computer system,but may be configured as a service provided using a cloud technology,and may be configured in some of the information display devices 130.

FIG. 7 schematically illustrates a functional configuration of theinformation management center 120. The information management center 120illustrated in FIG. 7 includes individual functional modules of acommunication section 701, a collected information analysis section 702,an information providing section 703, an image database 704, and a userdatabase 705. Each of the functional modules may be configured bydedicated hardware, or may also be achieved by executing predeterminedprograms on a computer system.

The communication section 701 performs communication with each of theimage capturing devices 110-1, 110-2, 110-3, . . . , and each of theinformation display devices 130-1, 130-2, 130-3, . . . , through a widearea network, such as the Internet, or the like.

The collected information analysis section 702 analyzes informationtransmitted from each of the image capturing devices 110-1, 110-2,110-3, . . . . Specifically, when the collected information analysissection 702 analyzes an image of a dangerous place, the collectedinformation analysis section 702 stores the image into the imagedatabase 704 in association with positional information. If imagecaptured time, a user-ID, an information tag for identifying a dangercategory, and the like are transmitted together with the image, thecollected information analysis section 702 also stores these pieces ofinformation together into the image database 704.

FIGS. 8A, 8B, and 8C illustrate an example of a data structure formanaging images that were collected from each of the image capturingdevices 110-1, 110-2, 110-3, . . . , in the image database 704. FIG. 8Aillustrates a basic data structure. One record in the image database 704is formed by a combination image data 801, which is a moving image or astill image produced by capturing a dangerous place, and positionalinformation 802 for identifying the site thereof. The image data 801 maynot be the entity of data, but may be a location (an image sharing site,or the like) for storing the image data, or a URL or a URI thatindicates accessing information. For example, a URL of the image contentuploaded to an image sharing site may be entered in the field of theimage data 801. Also, the notation method of the positional information802 is not limited in particular as long as it is suitable for matchingprocessing. For example, a latitude and a longitude, or an address maybe given.

FIG. 8B and FIG. 8C illustrate variations of a data structure formanaging an image in the image database 704, respectively. In theexample illustrated in FIG. 8B, fields of time 803 and user-ID 804 arefurther added to the data structure illustrated in FIG. 8A. Also, in theexample illustrated in FIG. 8C, a field indicating a danger category 805is added.

Referring back to FIG. 7 again, a description will be given of theconfiguration of the information management center 120 next. Thecollected information analysis section 702 performs image analysis onthe images collected from the individual image capturing devices 110-1,110-2, 110-3, . . . . Specifically, by image analysis, a determinationis made as to whether a dangerous place depends on time (for example,whether it is a dangerous place where a suspicious person is liable toappear after sunset, or it is a dangerous place by congestion during therush hours) is made by the environmental information or the informationtag (described above) attached to the image. Alternatively,identification may be made of a category (is it an accident, anincident, or a minor incident?) of a danger that the user who is aprovider of the image has encountered. These analysis results may bestored in the image database 704 in association with the image.

The user database 705 manages user information of each of the imagecapturing devices 110-1, 110-2, 110-3, . . . , and each of theinformation display devices 130-1, 130-2, 130-3, . . . . FIG. 9illustrates an example of a data structure for managing users in theuser database 705. In the example illustrated in FIG. 9, one record inthe user database 705 includes individual fields of a user's personalattribute (gender, age, character, occupation, and the like) 902, and auser attribute information 903 including a behavioral attribute, and thelike, and these fields are managed in association with a user-ID 901.

The information providing section 703 is allowed to obtain informationon the current position of each of the information display devices130-1, 130-2, 130-3, . . . , through the communication section 701.Then, the information providing section 703 searches the image database704 for a record corresponding to the current position of each of theinformation display devices 130-1, 130-2, 130-3, . . . . If thecorresponding record is found, the information providing section 703transmits the image stored in the record of the dangerous place, orinformation on the dangerous place, obtained by the image analysis, tothe information display device 130.

Also, the information providing section 703 may perform matching notonly using the current position, but may perform filtering with thepersonal attribute or the behavioral attribute of a user who uses eachof the information display devices 130-1, 130-2, 130-3, . . . , usingthe user database 705, and then may transmit an image or informationregarding a dangerous place obtained by image analysis if onlynecessary. This is because there are sites that are dangerous to women,and but not to men, sites that are dangerous to infants or elderlypeople in particular, and there are cases where it is dangerous whenrunning, but not dangerous in particular and unnecessary to payattention when walking, and the like. If information is transmitted to auser who is not in danger, it becomes a useless warning, and thusinformation will not be accepted seriously in the case of a real danger.

The information display device 130 displays information to moving users(of course, it is possible to display information to a user who remainsindoors). The information display device 130 is a multifunctional mobileterminal, such as a head mounted display, a smart phone, and the like,or a head up display which outputs enlarged information in a visualrange, such as a windshield of a vehicle, and the like, and which isallowed to communicate with the information management center 120through a wide area network, such as the Internet.

FIG. 10 schematically illustrates a functional configuration of theinformation display device 130. The information display device 130illustrated in FIG. 10 includes each of functional modules of a controlsection 1001, a communication section 1002, a positional informationacquisition section 1003, an image processing section 1004, a displaydrive section 1005, a display section 1006, a drive control section1007, and an actuator section 1008. Each of the functional modules maybe configured by dedicated hardware, and may also be achieved byexecuting predetermined programs on a computer system.

The control section 1001 includes a ROM 1001A and a RAM 1001B. The ROM1001A stores the program code executed by the control section 1001, andvarious kinds of data. The control section 1001 executes the programloaded in the RAM 1001B so as to totally control overall operation ofthe information display device 130, which includes display control ofimages. As examples of the program and the data that are stored in theROM 1001A, it is possible to give a communication control program withexternal devices, such as the information management center 120, and thelike, a display control program of images (of a dangerous place) thatare received from the information management center 120, a feedbackcontrol program in accordance with display of a dangerous place that theuser of the information display device 130 has encountered, specificidentification information to information display device 130, and userattribute information (described above) of the user who is using theinformation display device 130, and the like.

The positional information acquisition section 1003 obtains informationon the current position or posture of the information display device 130or the user. The positional information acquisition section 1003 mayinclude any one of or a combination of two or more sensors inconsideration of merits and demerits of each sensor of a gyro sensor, anacceleration sensor, a GPS sensor, a geomagnetic sensor, a Dopplersensor, an infrared sensor, a radio field intensity sensor, and the likein order to obtain position and posture information. Also, thepositional information acquisition section 1003 may use a combination ofinformation provided from various kinds of infrastructure, such asmobile-phone base station information, PlaceEngine (registeredtrademark) information, and the like in order to obtain the position andposture information.

The control section 1001 transmits the position and posture informationobtained by the positional information acquisition section 1003 to theinformation management center 120 through the communication section1002. Also, when the control section 1001 transmits the position andposture information, the control section 1001 may read the userattribute information from the ROM 1001A as necessary, and may transmitthe user attribute information together.

As already described, the information management center 120 searches theimage database 704 on the basis of the received positional information.If a matched dangerous place is found, the information management center120 transmits the captured image of the place, or the informationregarding the dangerous place obtained by image analysis to theinformation display device 130. Also, the information management center120 sometimes performs filtering with a personal attribute or abehavioral attribute of a user who uses each of the information displaydevices 130 using the user database 705, and then transmits an image orinformation regarding the dangerous place obtained by image analysis ifonly necessary.

Then, the control section 1001 presents the image provided from theinformation management center 120 to the user through the displaysection 1006. Specifically, the image processing section 1004 furtherperforms signal processing, such as image quality correction, and thelike on the image signal output from the control section 1001, andperforms resolution conversion to meet the screen of the display section1006. Then, the display drive section 1005 selects the pixels of thedisplay section 1006 for each line in sequence, performs line sequentialscanning, and supplies a pixel signal based on the image signal havingbeen subjected to the signal processing.

If the information display device 130 is a head mounted display, or ahead up display, information is displayed by being overlaid on adangerous place in the visual range of the user so as to effectivelynotify the user of a danger in order to avoid the danger.

Also, the control section 1001 may further call user's attention bygiving sound output, such as sound effect, or the like, vibration,performing tactile feedback in addition at a time when displaying thecaptured image of a dangerous place at the display section 1006. Theactuator section 1008 includes one or more output device groups forphysically outputting feedback to the user. The control section 1001controls operation of the actuator section 1008 through the drivecontrol section 1007. Note that the drive control section, and theactuator section 1008 are optional components.

FIG. 11 schematically illustrates an example of a configuration of theactuator section 1008. The actuator section 1008 includes at least someof output devices among a vibration device 1101, a heat generationdevice 1102, a cooling device 1103, an air sending device 1104, such asa fan, or the like, an acoustic device 1105, such as a speaker, or thelike, a light emitting device 1106, and a movement device 1107. Thecontrol section 1001 is capable of controlling operation of the outputdevices 1101 to 1107 through the drive control section 1007. Then, theactuator section 1008 is capable of generating vibration, heat, cooling,outputting wind, sound, emitting light, outputting pulses, and the likein addition to displaying an image of a dangerous place. In the casewhere the actuator section 1008 includes a large variety of outputdevices, and a plurality of output devices are operated in combination,it is possible to present various kinds of feedback.

Next, a description will be given of processing operation for providingdistribution services of information regarding a dangerous place in theinformation distribution system 100 according to the present embodiment.The information distribution services is roughly divided into an imageacquisition phase in which each of the image capturing devices 110-1,110-2, 110-3, . . . , obtains an image of a dangerous place, and aninformation providing phase in which the information management center120 collects the images obtained by each of the image capturing devices110-1, 110-2, 110-3, . . . , and classifies and provides the images tonecessary information display devices 130-1, 130-2, 130-3, . . . .

FIG. 12 is a flowchart illustrating a general processing procedure forcapturing an image of a dangerous place, which is executed in each ofthe image capturing devices 110-1, 110-2, 110-3, . . . . Note that it isassumed that the image capturing devices 110-1, 110-2, 110-3, . . . ,continue to obtain the current position and posture information all thetime during the processing operation.

In the image capturing device 110, a determination is made as to whetherthe device 110 or the user thereof is in a dangerous state or not (stepS1201). In this step, for example, the level of a danger is output as anindex value, such as a degree of danger, or the like.

Next, a determination is made as to whether it is timing to capture animage of a dangerous place on the basis of the current degree of dangerof the device 110 or the user thereof (step S1202). Here, if the degreeof danger of the device 110 or the user thereof is high, a determinationis made that the image capture timing has come (Yes in step S1202), andthe image capturing device 110 executes capturing of a moving image or astill image using the outside camera 612 at the current position (stepS1203).

Then, the image capturing device 110 generates transmission data byadding the current position and posture information to the image of thedangerous place captured in step S1203 (step S1204), and transmits theimage and the information to the information management center 120through the communication section 605 (step S1205).

In this regard, in step S1204, the image capturing device 110 maygenerate the transmission data by adding image-captured timeinformation, a user-ID of the user of the image capturing device 110, oruser attribute information in addition to the current position andposture information. Also, the image capturing device 110 may furtheradd the dangerous state information used for the determination of adanger in step S1201, or an information tag for determining the type ofa danger in addition to the transmission data.

FIG. 13 is a flowchart illustrating determination processing of adangerous state, which is executed by the image capturing device 110 instep S1201.

The control section 601 obtains biological information, such as pulse,perspiration, myoelectric potential, movement of eyeball, and the likefrom the state information acquisition section 604 (S1301). Then, thecontrol section 601 automatically determines whether the obtainedbiological information indicates that the user has encountered adangerous place or not (step S1302).

When physiological phenomena in which a user is in a state of extremetension is detected from biological information, for example, when theheart rate has abruptly increased, perspiration has been detected,muscle has abnormally contracted, eyes keep going out of focus or keep aclose watch on one place, or the like, it is possible for the controlsection 601 to determine that the user has encountered a dangerousplace. In this regard, it is possible to perform component analysis onsweat so as to identify the sweating is caused by body temperatureregulation or tension in order to increase determination precision.Also, in step S1302, more advanced and complicated determinationprocessing, for example, to what extent, is it dangerous relatively, orthe like may be performed in place of binary determination of whether itis dangerous or not by comparison of biological information with athreshold value. In order to perform such advanced and complicateddetermination processing, it is necessary to have a large amount ofdata, or a massive calculation load is imposed, and thus it is assumedto be difficult for the image capturing device 110 configured as a headmounted display to achieve the determination processing alone.Accordingly, cloud technology provides services that provide informationon the relative degree of danger indicated by biological informationbased on a large amount of recorded data, and the control section 601may use the information provided from such services.

Also, the control section 601 may perform determination processing instep S1302 in consideration of acceleration information together withthe biological information. When a high acceleration or speed wasimposed on the image capturing device 110 or the user, and then theheart rate has abruptly increased, it is possible to determine that aminor incident has occurred because the user stumbled on an obstacle ora hollow on the road.

Then, if determined that the user has encountered a dangerous place (Yesin step S1302), the control section 601 outputs a higher degree ofdanger on the basis of the pulse rate of the user or the amount ofperspiration or the components of sweat, or the like at that time (stepS1303). On the other hand, if determined that the user is not in adangerous place (No in step S1302), the control section 601 outputs alow degree of danger on the basis of the pulse rate of the user or theamount of perspiration or the components of sweat, or the like (stepS1304).

In this regard, in this processing routine, information of a dangerousstate used for determination of a danger, or an information tag fordetermination of a danger type may be output in addition.

FIG. 14 is a flowchart illustrating a variation of the determinationprocessing of a dangerous state, which is executed by the imagecapturing device 110 in step S1201. In the present embodiment, the imagecapturing device 110 is configured as a head mounted display, and it ispossible for the user to view a content, such as a movie, or the like bythe head mounted display. Accordingly, the user sometimes has beenemotionally involved in a viewing content, and as a result, a change mayoccur in the biological information, such as the pulse rate, theperspiration, and the like. This might bring about an erroneousdetermination of a dangerous state. Thus, the processing procedure inFIG. 14 is different from the processing procedure illustrated in FIG.13 in the point that determination of the degree of danger is changeddepending on whether the user is viewing a content or not.

The control section 601 obtains biological information, such as pulse,perspiration, myoelectric potential, movement of eyeball, and the likefrom the state information acquisition section 604 (S1401). Then, thecontrol section 601 automatically determines whether the obtainedbiological information indicates that the user has encountered adangerous place or not (step S1402).

Here, if the control section 601 determines that the user is not at adangerous place from the biological information (No in step S1402), thecontrol section 601 outputs a low degree of danger in accordance withthe biological information, such as the pulse rate of the user at thattime, the amount of perspiration, or the component of sweat, and thelike (step S1406).

On the other hand, if the control section 601 determines that the userhas encountered a dangerous place on the basis of the biologicalinformation (Yes in S1402), the control section 601 temporarily sets ahigher degree of danger (step S1403), and then further determineswhether the user is viewing a content at that time or not (step S1404).

If the user is not viewing a content (No in step S1404), the controlsection 601 directly outputs a higher degree of danger. On the otherhand, if the user is viewing a content (Yes in step S1404), a change ofthe user's biological information is a result of emotional involvementin a viewing content. Accordingly, there is a possibility that a dangerdoes not exist in reality, and thus the control section 601 decreases adegree of danger, and outputs the degree of danger (step S1405).

In this regard, in this processing routine, the control section 601 mayoutput information of a dangerous state used for determination of adanger, or an information tag for determining the type of the danger (asdescribed above).

Also, FIG. 15 is a flowchart illustrating another variation of thedetermination processing of a dangerous state, which is executed by theimage capturing device 110 in step S1201. In the determinationprocessing illustrated in FIG. 14, the impact on a change in biologicalinformation, which is accompanied by the user viewing a content isremoved. However, whether a change in the user's biological informationis due to a danger that occurred in a real world or viewing a content isstill unclear. Thus, in the processing procedure illustrated in FIG. 15,a determination of a degree of danger is made further in considerationof the environmental information of the image capturing device 110 orthe user with higher precision.

The control section 601 obtains biological information, such as pulse,perspiration, myoelectric potential, movement of eyeball, and the likefrom the state information acquisition section 604 (S1501). Then, thecontrol section 601 automatically determines whether the obtainedbiological information indicates that the user has encountered adangerous place or not (step S1502).

Here, if the control section 601 determines that the user is not at adangerous place from the biological information (No in step S1502), thecontrol section 601 outputs a low degree of danger in accordance withthe pulse rate of the user at that time, the amount of perspiration, orthe components of sweat, and the like (step S1509).

On the other hand, if the control section 601 determines that the userhas encountered a dangerous place on the basis of the biologicalinformation (Yes in S1502), the control section 601 temporarily sets ahigher degree of danger (step S1503), and then further determineswhether the user is viewing a content at that time or not (step S1504).

If the user is not viewing a content (No in step S1504), the controlsection 601 directly outputs a higher degree of danger. On the otherhand, if the user is viewing a content (Yes in step S1504), there is apossibility that a change of the user's biological information is aresult of emotional involvement in a viewing content. Thus, the controlsection 601 obtains information on the environment in which the imagecapturing device 110 or the user is placed from the environmentalinformation acquisition section 616 (step S1505), and automaticallydetermines whether the environmental information indicates that the userhas encountered a dangerous place (step S1506).

For example, if the control section 601 has determined, from theobtained environmental information, that a big sound, such as noises, anexplosive sound, or the like was detected, that a strong wind, such as astorm, an air blast, or the like was detected, that a high temperaturelike a fire location, such as a fire site, a vehicle collision accident,or the like was detected, an atmosphere (for example, there is a thickfog or smoke, radioactive rays, and a toxic chemical substance in theatmosphere), or the like is detected, it is possible to determine thatthe user has encountered a dangerous place. Also, when it is recognizedthat the user is at a site of an accident, an incident, or the like byanalyzing the image captured by the outside camera 612, it is possibleto determine that the user has encountered a dangerous place.

Here, if the control section 601 has determined that the userencountered a dangerous place on the basis of the environmentalinformation (Yes in step S1506), the control section 601 directlyoutputs a higher degree of danger. Also, even if a determination is madethat the user did not encounter a dangerous place on the basis of theenvironmental information (No in step S1506), when the user inputs aninstruction stating that it is dangerous through the input operationsection 602, a remote control, or the like (Yes in step S1507), thecontrol section 601 directly outputs a higher degree of danger.

On the other hand, if a determination is made that the user did notencounter a dangerous place on the basis of the environmentalinformation (No in step S1506), and the user did not input aninstruction stating that it is a dangerous place (No in step S1507), thecontrol section 601 has determined that the image capturing device 110or the user did not encounter a dangerous place, reduces the degree ofdanger, and then outputs the degree of danger (step S1508).

In this regard, in this processing routine, the control section 601 mayoutput information of the dangerous state used for determination of adanger, or the information tag for determining the type of the danger(as described above).

If a person encounters a phenomenon, such as a sound, that causes theperson to predict a danger (for example, an engine sound of anautomobile approaching rapidly, a brake squeal, a car horn, headlight,wind (abrupt movement of air), vibration, and the like), the personautomatically gives his/her line of sight to that direction. Then, whenthe person witnesses the phenomenon, that is to say, a dangerous state,his/her experiences a biological change, such as an increase in pulse,and perspiration. Accordingly, it is possible to correctly determine adegree of danger with high precision using the person's behavioralcharacteristic and physiological phenomena.

FIG. 23 is a flowchart illustrating a processing procedure fordetermining a dangerous state using behavior characteristic andphysiological phenomena of a person.

First, the control section 601 identifies a place where the user of theimage capturing device 110 predicts a danger (step S2301).

In step S2301, the control section 601 identifies a place that causesthe user to predict a danger on the basis of the environmentalinformation obtained from the environmental information acquisitionsection 616. Specifically, the control section 601 identifies a soundsource of a big sound, a windward, such as air blast, or the like, alight source of strong light, a source of vibration, and the like.

Next, the control section 601 extracts information on the user'sposition and posture information and the line of sight from thebiological information (for example, electro oculograpy, and at leastone of the images of left and right eyes, captured by the inside camera)received from the state information acquisition section 604, and obtainsthe direction of line of sight (step S2302).

Then, the control section 601 checks whether the user's line of sightfalls on (or substantially falls on) the place identified in step S2301to predict a danger (step S2303).

Here, if the user's line of sight falls on (or substantially falls on)the place identified in step S2301 to predict a danger (Yes in stepS2303), the control section 601 further obtains biological information,such as pulse, perspiration, myoelectric potential, movement of eyeball,and the like from the state information acquisition section 604 (S2304).Then, the control section 601 automatically determines whether theobtained biological information indicates that the user has encountereda dangerous place or not (step S2305).

When physiological phenomena such that a user is in a state of extremetension is detected from biological information, for example, heart ratehas abruptly increased, perspiration has been detected, muscle hasabnormally contracted, eyes keep going out of focus or keep a closewatch on one place, or the like, it is possible for the control section601 to determine that the user has encountered a dangerous place. Inthis regard, it is possible to perform component analysis on sweat so asto identify the sweating is caused by body temperature regulation ortension in order to increase determination precision. Also, when a highacceleration or speed was imposed on the image capturing device 110 orthe user, and then the heart rate has abruptly increased, it is possibleto determine that a minor incident has occurred because the userstumbled on an obstacle or a hollow on the road.

Then, if determined that the user has encountered a dangerous place (Yesin step S2305), the control section 601 outputs a higher degree ofdanger on the basis of the pulse rate of the user or the amount ofperspiration or the components of sweat, or the like at that time (stepS2306). On the other hand, if determined that the user is not in adangerous place (No in step S2305), the control section 601 outputs alow degree of danger on the basis of the pulse rate of the user or theamount of perspiration or the components of sweat, or the like (stepS2307).

In this regard, in this processing routine, the control section 601 mayoutput information of a dangerous state used for determination of adanger, or an information tag for determining the type of the danger (asdescribed above).

As described above, it is possible to correctly determine whether theuser has encountered a dangerous place in combination of the user'sbiological information, the viewing state of a content, and theenvironmental information in which the image capturing device 110 or theuser is in. Also, the image capturing device 110 is configured as a headmounted display, and is used by being worn on the head of the user, andthus it is possible to automatically capture an image in the user'sdirection of line of sight using the outside camera 612. However, a userwho has encountered a dangerous place does not necessarily continue towatch the dangerous place. Accordingly, there is a possibility that animage of a thing not related to a danger at all is captured, and istransmitted to the information management center 120. Thus, the imagecapturing device 110 may determine whether the user is watching adangerous place on the basis of the user's biological informationobtained by the state information acquisition section 604, or theenvironmental information obtained by the environmental informationacquisition section 616, or the like, and may perform image capturingoperation.

FIG. 16 is a flowchart illustrating a processing procedure for capturingan image of a dangerous place, which is executed by the image capturingdevice 110 in step S1203. Note that it is assumed that a still image ofa dangerous place is captured in the processing procedure illustrated inFIG. 16.

The control section 601 extracts the user's position and postureinformation and the line-of sight information from the biologicalinformation received from the state information acquisition section 604(step S1601).

Next, the control section 601 checks whether the user's line of sighremains at the same place for a certain period of time or not (stepS1602).

Then, if it is detected that the user's line of sight remained at thesame place for a certain period of time (Yes in step S1602), it isestimated that the user gazes at an accident site, an incident site, ora place where a minor incident has occurred. Accordingly, the controlsection 601 cause the outside camera 612 to direct to the user'sdirection of line of sight, and to capture a still image (step S1603).

Also, FIG. 17 is a flowchart illustrating another processing procedurefor capturing an image of a dangerous place, which is executed by theimage capturing device 110 in step S1203. Note that it is assumed that amoving image of a dangerous place is captured in the processingprocedure illustrated in FIG. 17.

The control section 601 extracts the user's position and postureinformation and line-of-sight information from the biologicalinformation received from the state information acquisition section 604(step S1701).

Next, the control section 601 checks whether the user's line of sightremained at the same place for a certain period of time (step S1702).

Then, if it is detected that the user's line of sight remained at thesame place for a certain period of time (Yes in step S1702), it isestimated that the user gazes at an accident site, an incident site, ora place where a minor incident has occurred. Accordingly, the controlsection 601 cause the outside camera 612 to direct to the user'sdirection of line of sight, and to start capturing a moving image (stepS1703).

After that, the control section 601 continues to detect the user's lineof sight while capturing the image (step S1704). Then, if the user'sdirection of line of sight changes a certain amount or more (Yes in stepS1705), it is estimated that the user's line of sight falls off theaccident site, the incident site, or minor incident place, and thus thecontrol section 601 stops capturing the moving image (step S1706).

FIG. 18 is a flowchart illustrating still another processing procedurefor capturing an image of a dangerous place, which is executed by theimage capturing device 110 in step S1203. In the processing procedureillustrated in FIG. 16 and FIG. 17, a determination as to whether theuser watched only a dangerous place or not is made only by the directionof line of sight. In contrast, the processing procedure illustrated inFIG. 18 is different from the above in the point that a dangerous placeis identified on the basis of the environmental information, and thelike that is obtained by the environmental information acquisitionsection 616. Also, in the processing procedure illustrated in FIG. 18,it is assumed that a moving image of a dangerous place is captured.

First, the control section 601 identifies a dangerous place in thevicinity of the image capturing device 110 or the user (step S1801).

In step S1801, the control section 601 identifies the dangerous place onthe basis of the environmental information obtained from theenvironmental information acquisition section 616, for example.Specifically, the control section 601 identifies a dangerous place, forexample, a sound source of a big sound, such as a noise or an explosivesound, a windward of a strong wind, such as a storm or an air blast, orthe like, a fire site based on temperature distribution or a firelocation of a vehicle collision accident, or the like, an occurrenceplace of a thick fog or smoke, a source of radioactive rays, a source oftoxic chemical substance, and the like. Also, the control section 601may identify the image captured by the outside camera 612, andidentifies a dangerous place, such as a site of an accident or anincident, or the like, a site of a minor incident, or the like.

Alternatively, the control section 601 may receive information on anaccident or an incident that has occurred in the vicinity, and the likefrom the outside through the communication section 605, and may identifya neighboring dangerous place on the basis of the received information.

Next, the control section 601 extracts information on the user'sposition and posture information and line of sight from the biologicalinformation (for example, electro oculograpy, and at least one of theimages of left and right eyes, captured by the inside camera) receivedfrom the state information acquisition section 604, and obtains thedirection of line of sight (step S1802).

Then, if the user's line of sight falls on (or substantially falls on)the dangerous place identified in step S1801 (Yes in step S1803), thecontrol section 601 starts capturing a moving image with the outsidecamera 612 directed to the user's direction of line of sight (stepS1804).

After that, the control section 601 continues to detect the user's lineof sight during an image capturing period (step S1805). Then, when theuser's line of sight falls off the dangerous place (Yes in step S1806),the control section 601 stops capturing the moving image (step S1807).

Also, FIG. 19 is a flowchart illustrating still another processingprocedure for capturing an image of a dangerous place, which is executedby the image capturing device 110 in step S1203. In the processingprocedure illustrated in FIG. 19, a dangerous place is identified on thebasis of the environmental information obtained by the environmentalinformation acquisition section 616, and the like in the same manner asthe processing procedure illustrated in FIG. 18. However, the processingprocedure illustrated in FIG. 19 is different in the point that whethera moving image or a still image is captured depends on the level ofimportance of a dangerous place or its type. This brings aboutadvantages in that when an important accident or incident occurs, or inthe case where a still image fails to convey the state of a dangerousplace, it is possible to provide a moving image, and in the other cases,it is possible to reduce the amount of transmission data by providing astill image.

First, the control section 601 identifies a dangerous place in thevicinity of the image capturing device 110 or the user (step S1901).

The method of identifying a dangerous place on the basis of theenvironmental information is the same as that described with referenceto FIG. 18. At this time, the level of importance and the type of adanger of the identified place is determined together, and adetermination is made as to the method of capturing an image, that is tosay, whether a moving image or a still image is to be captured on thebasis of the determination result (step S1902).

Next, the control section 601 extracts the user's position and postureinformation, and line-of-sight information from the biologicalinformation received from the state information acquisition section 604,and checks the direction of line of sight (step S1903).

Then, if the user's line of sight falls on (or substantially falls on)the dangerous place identified in step S1901 (Yes in step S1904), thecontrol section 601 captures an image.

Here, if it is determined to capture a still image in step S1902 (Yes instep S1905), the outside camera 612 is directed to the user's directionof line of sight, captures a still image (step S1906), and then thisprocessing routine is terminated.

On the other hand, if determined to capture a moving image in step S1902(No in step S905), the control section 601 directs the outside camera612 to the user's direction of line of sight, and starts capturing amoving image (step S1907). Also, the control section 601 continues todetect the user's line of sight during an image capturing period (stepS1908). Then, if the user's direction of line of sight falls off theaccident site, the incident site, or, the place of the minor incident(Yes in step S1909), the control section 601 stops capturing the movingimage (step S1910), and terminates this processing routine.

For example, if a pedestrian encounters an engine sound of an automobileapproaching rapidly, a brake squeal, a car horn, headlight, wind (abruptmovement of air), vibration, and the like, the person automaticallygives his/her line of sight to that direction. If the pedestrian iswearing a head mounted display as an image capturing device 110, it ispossible for the pedestrian to immediately catch a state in thepedestrian's direction of line of sight in accordance with theabove-described processing procedure (without the pedestrian beingconscious of), and to capture the image of a dangerous place. That is tosay, with a technique disclosed in this specification, it is possiblefor a person who happens to be at a dangerous place to capture an imagein a situation where a danger is approaching. On the other hand,although it is possible to use a multifunctional terminal with a cameracarried by a pedestrian, such as a smart phone, or the like to use animage capturing device 110, it becomes necessary for the pedestrian totake a conscious action or operation to direct a camera to the viewingdirection in order to capture the image of the dangerous place in thesame situation. Accordingly, it is difficult for the pedestrian toimmediately take a reflective action. Also, if the pedestrian carriesthe multifunctional terminal in a pocket of clothes or in a bag, it isdifficult to immediately correspond to the situation where a dangeroccurs or when encountering a dangerous place (or it is necessary toprepare in advance all the time), and thus the pedestrian is liable tolose an image capturing timing at an important situation.

With the image capturing device 110 configured as a head mounteddisplay, it is possible to capture an image in a situation where adanger is approaching. However, it is possible to make a determinationof a degree of danger with a further higher precision. For example, itis possible to determine an approaching objet, such as an automobile, amotorcycle, a bicycle, and the like. When a moving image is captured, itis possible to calculate an approaching speed of an object. Also, aftercollision, it is possible to determine the damage on the basis of theacceleration imposed by the collision. Also, when accidents frequentlyoccurred, or when accidents nearly happened, but were prevented (only anincrease in the heart rate, but no damage by the collision or the like),it is possible to determine from what point it is really dangerous, andto provide analysis information for the improvement.

Also, when the image capturing device 110 is configured as a headmounted display, it is possible to identify a degree of danger from theviewpoint of each user who wears the device. For example, it isdifficult for a child to go near an approaching car, but for a talladult, it is sometimes not dangerous. Also, an obstacle or a hollow onthe sidewalk is not so dangerous for a person with good eyesight, but isdangerous for a person with poor eyesight. The same dangerous place hasa different degree of danger depending on attributes, such as an age,gender, a physical characteristic, a character, and the like. It ispossible for the image capturing device 110 as a head mounted display toobtain an image of a dangerous place in consideration of an age, gender,a physical characteristic, a character, and the like on the basis of achange in biological information based on the viewpoint of each user.

To date, there have been cameras capable of capturing dangerous scenesthat occur at street corners, such as a monitoring camera, a securitycamera. However, these fixed-point cameras are disposed at danger zonesthat are mainly notorious places so far. However, it is rare that thesefixed-point cameras are disposed at latent danger zones (to put itanother way, places that are unclear as to whether a danger occurs ornot). It costs too much to dispose fixed-point cameras at latent dangerzones in terms of installation of the facilities. In contrast, with thetechnique disclosed in this specification, the image capturing device110 is an imaging device that is carried by the user as a head mounteddisplay, and thus it is possible to solve the inconvenience not allowedto install the facilities.

In addition, with the information distribution system 100 that capturesan image of a dangerous place on the assumption that a large number ofusers are wearing a head mounted display, unlike fixed-point cameras, itbecomes possible to give a deterrent force to a criminal to commit acrime, and to improve public safety and security of the users.

Each of the image capturing devices 110-1, 110-2, 110-3, . . . ,captures an image of a dangerous place in accordance with any one of theprocessing procedures illustrated in FIG. 16 to FIG. 19, then in thefollowing step 1204, creates transmission data by adding the positionalinformation of the site, and transmits the image and the data to theinformation management center 120. Also, to the transmission data, eachimage capturing device may further attach information, such as time ofthe encounter with the danger, information regarding an encountereddanger category (an accident, an incident, or a minor incident) (or aninformation tag to be used for identifying a danger category), userattribute information including a personal attribute of the user(gender, an age, a character, an occupation, and the like), a behavioralattribute, and the like.

The information management center 120 manages images of the dangerousplace that have been received from each of the image capturing devices110-1, 110-2, 110-3, . . . , in association with the positionalinformation. Also, the information management center 120 may classifythe images provided from each of the image capturing devices 110-1,110-2, 110-3, . . . , for each danger category as necessary, and maymanage the images in association not only with the positionalinformation, but also with the other information, such as the userattribute information (described above), and the like.

FIG. 20 is a flowchart illustrating a procedure for the informationmanagement center 120 to process an image of a dangerous place receivedfrom each of the image capturing devices 110-1, 110-2, 110-3, . . . .

When the collected information analysis section 702 receives data fromany one of the image capturing devices 110-1, 110-2, 110-3, . . .through the communication section 701 (Yes in step S2001), the collectedinformation analysis section 702 analyzes received data, extracts theimage of the dangerous place and the positional information of the site(step S2002), and creates a record including the image of the dangerousplace and the positional information in association with each other(step S2003).

Next, the collected information analysis section 702 checks whether thereceived data includes information of the image captured time of thedangerous place (step S2004). If the information is included (Yes instep S2004), the collected information analysis section 702 adds thetime information to the above-described record (step S2005).

Also, the collected information analysis section 702 checks whether itis possible to identify a user of the image capturing device 110, whichbecomes the data transmission source (step S2006). The method ofidentifying the user includes a case where the transmission sourceaddress included in the received data and the user are managed uniquely,a case where the received data includes a user-ID, or the like. In thecase where it is possible to identify the user (Yes in step S2006), thecollected information analysis section 702 adds the user-ID of the userto the above-described record (step S2007).

Also, the collected information analysis section 702 checks whether thereceived data includes an information tag for identifying a dangercategory (step S2008). If the information tag is included (Yes in stepS2008), the collected information analysis section 702 analyzes theinformation tag, identifies the danger category, and adds the dangercategory to the above-described record (step S2009).

Then, the collected information analysis section 702 stores the createdrecord into the image database 704 (step S2010), and terminates thisprocessing routine. The data structure of the record in the imagedatabase 704 is illustrated in FIG. 8.

It is possible for the information management center 120 to provide ahazard map service by mapping information of the dangerous placescollected from a large number of image capturing devices 110-1, 110-2,110-3, . . . onto a map. The information providing section 703 maycreate a hazard map by mapping information of dangerous places includingall the danger categories onto the map. Alternatively, the informationproviding section 703 may create a hazard map for each danger category(for example, a hazard map of traffic accidents, a hazard map of fires,a hazard map of sexual molestation or assault incidents, and the like).

Also, the information management center 120 may distribute the createdhazard map to each of the information display devices 130-1, 130-2,130-3, . . . , or may upload and publish the hazard map on an imagesharing site.

As already described, images are collected from the individual imagecapturing devices 110-1, 110-2, 110-3, . . . , with identification of adegree of danger in accordance with a viewpoint of each user. Thus, inthe information management center 120, it is possible for theinformation providing section 703 to provide a higher-precision hazardmap that is customized to each of the user's attributes, such as an age,gender, a physical characteristic, a character, and the like. Forexample, it is possible to achieve delicate services suited to adifference of attribute for each user as follows.

(1) When an infant is conveyed in a baby carriage, it may be dangerousat a gap between a sidewalk and a road, because a contact with a car isliable to occur.

(2) Little children or short elementary school kids are liable toencounter a danger of being touched on a face or an eye by a smokingcigarette by a smoking person while walking. A place where anunspecified large number of people are crossing, such as busy streets infront of a station, a pedestrian crossing at an intersection, and thelike may be dangerous.

(3) For short little children and elementary school kids, a desertedplace may be dangerous, because they are liable to be annoyed by akidnapper, a homeless person, and a suspicious person.

(4) Men may want to learn where they are liable to have trouble with agangster.

(5) Young women may want to learn where they are liable to encounter asexual molester.

(6) Regardless of age and gender, people may want to learn where is aconflict area of gangs, and where traffic accidents occurs frequently.

Also, the information management center 120 searches the image database704 with a current position of each of the information display devices130-1, 130-2, 130-3, . . . , and if a matched dangerous place is found,the information management center 120 transmits the captured image ofthat site, or the information regarding the dangerous place obtained bythe image analysis to the corresponding information display device 130.

FIG. 21 is a flowchart illustrating a processing procedure for theinformation management center 120 to transmit an image of a dangerousplace or the information thereof to each of the information displaydevices 130-1, 130-2, 130-3, . . . .

The information providing section 703 attempts to obtain the positionalinformation of each of the information display devices 130-1, 130-2,130-3, . . . (step S2101).

The method of obtaining the positional information of the informationdisplay device 130 is not limited in particular. The informationmanagement center 120 sometimes monitors the information on the currentposition of each of the information display devices 130-1, 130-2, 130-3,. . . , all the time. Also, an information display device 130 thatrequests information on a dangerous place sometimes transmits thepositional information of itself.

When it was possible for the information providing section 703 to obtainthe positional information of any one of the information display devices130 (Yes in step S2101), the information providing section 703 searchesthe image database 704 using the positional information (step S2102).

If there are one or more records that match the positional informationin the image database 704 (Yes in step S2103), filtering processing isperformed as necessary.

Specifically, if time information is described in the correspondingrecord (Yes in step S2104), whether the time information of the recordmatches the current time or not is further checked (step S2105). Therecord having the time information that does not match the current timeis handled as unmatched (No in step S2105).

Also, if a user-ID is described in the corresponding record (Yes in stepS2106), a search is made of the user database 705 using the user-ID toextract the user's personal attribute and behavioral attribute (stepS2107). Then, filtering is performed on the basis of the similarity tothe personal attribute and the behavioral attribute of the user of theinformation display device 130 to be the transmission target of theimage (step S2108).

The record including the image collected from the user whose personalattribute and behavioral attribute are not similar is handled asunmatched (No in step S2109). This is because there are sites that aredangerous to women, and but not to men, sites that are dangerous toinfants or elderly people in particular, and there are cases where it isdangerous when running, but not dangerous in particular and unnecessaryto pay attention when walking, and the like. Also, an obstacle or ahollow on the sidewalk is not so dangerous for a person with goodeyesight, but is dangerous for a person with poor eyesight. Ifinformation is transmitted to a user who is not in danger, it becomes auseless warning, and thus information will not be accepted seriously inthe case of a real danger.

Also, if a danger category is described in the corresponding record (Yesin step S2110), the danger category is extracted (step S2111).

Then, the information providing section 703 creates transmission data(step S2112). The data of the image remained after the filteringprocessing is included in the transmission data.

The transmission data may include the image data of a dangerous placeitself, or the location storing the image data (image sharing site, orthe like), or an URL or an URI that indicates the information to beaccessed.

Also, the information providing section 703 may not include the imagedata extracted from the image database 704 directly in the transmissiondata, but may transmit the image data suitably processed in accordancewith a device attribute, such as display ability of the informationdisplay device 130 of the transmission destination, or the userattribute, the danger category, and the like. For example, theinformation providing section 703 changes the resolution and the imagesize in accordance with whether the information display device 130 ofthe transmission destination is a head mounted display or a head updisplay. Also, if a user of the information display device 130 of thetransmission destination is an adult, the information providing section703 inserts a detailed message including Chinese characters, but if theuser is a little child, the information providing section 703 inserts amessage not including difficult Chinese characters, but including imagesthat are easy to understand, such as pictorial symbols, or the like.Also, if a danger category is an urgent case, such as a trafficaccident, a fire, or the like, the information providing section 703promotes awareness of the user by displaying in a reddish color. If itis a place where a suspicious person frequently appears, the informationproviding section 703 urges the user to avoid approaching by displayingin a dark color. In this regard, a part of all of the above-describedprocess processing of the image data may be performed by the informationdisplay device 130 of the transmission destination in place of theinformation management center 120.

On the other hand, if the record of the corresponding image is not foundin the image database 704 (No in step S2103), or if there remains noimage after the filtering processing (No in step S2105 or S2109), adescription stating that there is no corresponding image is written intothe transmission data in step S2112. Also, if a danger category isobtained in step S2111, the danger category may be included in thetransmission data. The created transmission data is transmitted to thecorresponding information display device 130 through the communicationsection 1002 (step S2113). In this regard, it is assumed that there area plurality of records of the corresponding image as a result of thefiltering processing on the basis of the time information and the user'sattribute information. In such a case, it is thought that there are twomethods of transmitting the data. One is a method in which theinformation management center 120 narrows down the records to one (orsome of the records) to transmit the data to the information displaydevice 130. The other is a method of transmitting all the correspondingimages to the information display device 130. For the details, adescription will be given later.

When each of the information display devices 130-1, 130-2, 130-3, . . ., receives an image of a dangerous place, or information regarding adangerous place obtained by image analysis from the informationmanagement center 120, the information display device displays the imageor the information so as to give a warning against a danger to the userwho is carrying the device.

For example, if the information display device 130 is a head mounteddisplay, or a head up display, information is displayed by beingoverlaid on a dangerous place in the visual range of the user so as toeffectively notify the user of a danger in order to avoid the danger.Also, the information display device 130 may further call user'sattention by giving sound output, such as sound effect, or the like,vibration, performing tactile feedback in addition at a time whendisplaying a captured image of a dangerous place.

FIG. 22 is a flowchart illustrating a processing procedure for theinformation display device 130 to display information regarding adangerous place, received from the information management center 120, toa user. However, it is assumed that in the processing procedure in FIG.22, a part of or all of the image filtering processing, and the processprocessing of the image data is performed by the information displaydevice 130 in place of the information management center 120.

When the control section 1001 receives data from the informationmanagement center 120 by the communication section 1002 (Yes in stepS2201), the control section 1001 decodes the received data, and extractsvarious data, such as the image of a dangerous, and the like (stepS2202). In this regard, if the received data does not include the imageof a dangerous place itself, but includes a URL or a URI, the image isdownloaded from the corresponding image sharing site, or the like instep S2202. Also, it is assumed that information on a plurality ofimages regarding a dangerous place at which the user of the informationdisplay device 130 is present is transmitted from the informationmanagement center 120.

Next, the control section 1001 checks whether the received data includesa danger category or not (step S2203). If a danger category is included(Yes in step S2203), the control section 1001 performs processing on theimage obtained in step S2202 in accordance with the danger category asnecessary (step S2204). For example, if a danger category is an urgentcase, such as a traffic accident, a fire, or the like, the controlsection 1001 promotes awareness of the user by displaying a dangerousplace in a reddish color. Also, if it is a place where a suspiciousperson frequently appears, the control section 1001 urges the user toavoid approaching by displaying a dangerous place in a dark color.

Next, the control section 1001 attempts to obtain the user attributeinformation of the information display device 130 from the ROM 1001A,the user database 705, or the like (step S2205). Then, if it waspossible to obtain the user attribute information (Yes in step S2206),the control section 1001 performs processing on the image obtained instep S2202 in accordance with the user's attribute information asnecessary (step S2207). For example, if the user is an adult, thecontrol section 1001 inserts a detailed message including Chinesecharacters, but if the user is a little child, the control section 1001inserts a message not including difficult Chinese characters, butincluding images that are easy to understand, such as pictorial symbols,or the like. Also, if information on a plurality of images istransmitted from the information management center 120, the controlsection 1001 performs filtering processing in accordance with the timeinformation, and the user's personal attribute and behavioral attributein step 2207.

Then, when the positional information acquisition section 1003 obtainsthe current position and posture information of the user of theinformation display device 130 (step S2208), the control section 1001displays the image of a dangerous place by following the user's headmovement (step S2209). If the information display device 130 is, forexample, a head mounted display or a head up display, the image of adangerous place is displayed in a see-through manner so as to beoverlapped on a dangerous place of a real scene. In this regard, ifinformation on a plurality of images is transmitted from the informationmanagement center 120, it is assumed that even if filtering processingis performed in step S2207, it is difficult to narrow down the images toone image. For the details of the processing at that time, a descriptionwill be given later.

FIG. 24 to FIG. 27 illustrate a display example of an image of adangerous place on a certain information display device 130. Note thatin each of FIG. 24 to FIG. 27, it is assumed that the informationdisplay device 130 is configured as a head mounted display, or a head updisplay, and the image of a dangerous place is displayed in anoverlapping manner on the scene (a street corner, or the like) in thedirection of line of sight of the user of the corresponding device.

The example illustrated in FIG. 24 illustrates a fact that if a rightturn is made at a street corner illustrated by reference numeral 2401,the user will find a dangerous place, which is illustrated by an object2402 that blocks the road. If a danger category is an urgent case, suchas a traffic accident, a fire, or the like, awareness of the user ispromoted by displaying in a reddish color in order for the user to payattention to pass through.

In the example illustrated in FIG. 25, a message stating “A trafficaccident has occurred!” is further displayed as illustrated by referencenumeral 2501 using a balloon in order to notify a danger category to theuser. In this regard, an easy-to-understand sentence may be used for themessage to meet the user's attribute information, such as the user'sage, gender, and the like, for example (for example, if the user is alittle child, or a lower-grade elementary school kids, Chinesecharacters are avoided, and sound message is used instead).

Also, the example illustrated in FIG. 26 illustrates a fact that if aright turn is made at a street corner illustrated by reference numeral2601, the user will find a dangerous place, which is illustrated by anobject 2602 that blocks the road. Unlike the example illustrated in FIG.24, in the case of a dangerous place that is to be avoided to passthrough in the first place, because if a right turn is made at a streetcorner 2601, a dangerous person to the user (for example, a suspiciousperson, a sex molester, a violent person, or a criminal) frequentlyappears, or the like, an object 2602 that covers the street corner 2601itself with darkness to be unseen is displayed. Of course, in the samemanner as the case illustrated in FIG. 25, a message indicating a dangercategory may be displayed together.

Also, FIG. 27 illustrates an example in which a display of a dangerousplace is applied to a navigation system. In the example in FIG. 27, theoriginal guidance is illustrated by a dashed-line arrow as illustratedby reference numeral 2701, and a guidance for avoiding the dangerousplace is illustrated by a solid-line arrow as illustrated by referencenumeral 2702. In order for the user not to be confused, the display ofthe original guidance 2701 may be omitted. Also, the original guidancemay be given by displaying a message using a balloon 2703 stating thereason why reference numeral 2701 became difficult to pass.

AS is understood from FIG. 24 to FIG. 27, with the techniques disclosedin this specification, it is possible to suitably provide an image of adangerous place including a source of an accident, and an incident.

It is assumed that a plurality of images are hit regarding a dangerousplace where the user of the information display device 130 is located.In the processing procedure illustrated in FIG. 21 or FIG. 22, it isassumed that filtering processing is performed in accordance with thetime information and the user's personal attribute and behavioralattribute, and thereby images are narrowed down to a predeterminednumber of images (examples in which images are narrowed down to oneimage are illustrated in FIG. 24 to FIG. 27). However, it is alsoassumed that a plurality of images are hit on the same condition of thetime information and the user's attribute information, and thus it isdifficult to narrow down images to a predetermined number of imagessimply by the filtering processing. In such a case, a plurality ofsearch conditions may be set in order to make a search, and images maybe narrowed down to a predetermined number of images. In order toperform such narrow-down processing, three methods (A) to (C) areconsidered as follows.

(A) The information management center 120 narrows down the images usinga default search condition.

(B) The images are narrowed down using a search condition specified bythe information display device 130.

(C) The information display device 130 obtains necessary informationfrom the information management center 120, and the images are narroweddown using a desired search condition.

Here, a description will be given of a scene in which the user of theinformation display device 130 is a young woman, and someone (asuspicious person) is chasing her as a specific example. She speculatesas to how she can avoid a danger of being driven into a dead-end street,or a danger of being attacked on both sides by another accomplice. Insuch a case, the following pieces of information (a) and (b) aredemanded.

(a) The best way to escape (a danger zone map within surrounding 1 km (adistance of 10-minute run))

(b) A face of an accomplice

As a search condition of an image in the case where a young womanbecomes a user includes the following (1) to (4), for example.

(1) Dangerous images that were experienced by women aged within threeyears from the user

(2) Images obtained within 30 minutes before and after the current timein the past two weeks

(3) An image including a man rather than women

(4) A maximum danger zone within surrounding 1 km (a distance of10-minute run) (or a place of an actual crime)

The images that satisfies the above-described search conditions (1) to(4) are extracted for a predetermined number of pieces (for example,five pieces) in the order of descending degree of danger, and arepresented to the user. Also, if the extracted number is less than thepredetermined number of pieces, it is preferable for the informationmanagement center 120 to relax the search conditions (1) to (3) as asearch condition (5) in order to adjust to the predetermined number ofpieces (for example, expand the age width from within three years towithin five years, expand the time from before and after 30 minutes fromthe current time to before and after one hour, expand a time period fromthe past two weeks to the past one month, and the like).

In the case of employing the above-described method (A), the informationdisplay device 130 whose user is a young woman transmits a requeststating that she was chased and dangerous so that the information is tobe transmitted. The information management center 120 narrows down theimages using the above-described default search conditions (1) to (4)from the user's attribute information to a predetermined number ofimages, and transmits the predetermined number of images to theinformation display device 130. Also, the information management center120 performs condition relaxation (5) as necessary.

Also, in the case of employing the above-described method (B), when theinformation display device 130 requests transmission of dangerousinformation, the information display device 130 specifies a desiredsearch condition out of the default search conditions provided by theinformation management center 120. For example, if the search conditions(1) to (3) are specified, the information management center 120 searchesfor an image in that condition, and provides the result to theinformation display device 130. Alternatively, the information displaydevice 130 may freely specify a search condition without beingrestricted by default search conditions. For example, theabove-described search condition (3) may be replaced by (3)′ two or morepersons are included (In the case where an accomplice is female, andmoney and goods are forcefully demanded), and the search conditions (1),(2), (3)′, (4), and (5) may be specified.

Also, in the case of employing the above-described method (C), theinformation display device 130 requests transmission of imagessatisfying the search conditions (1) and (2) from the informationmanagement center 120 in order to perform image search processing usingall the search conditions (1) to (5). Then, the information displaydevice 130 performs search processing on the images obtained from theinformation management center 120 to narrow down the images using thesearch conditions (3) to (5). In the case where the information displaydevice 130 makes a search again by relaxing the search conditions (3) to(5), even if it becomes not possible to communicate with the informationmanagement center 120, it is possible to perform search processing.Also, if the communication line with the information management center120 is narrow, there is an advantage in that the information displaydevice 130 is capable of obtaining a search result quickly.

In the above, a detailed description has been given of the techniquedisclosed in this specification with reference to specific embodiments.However, it will be obvious to those skilled in the art thatmodifications and substitutions may be made without departing from thespirit and scope of the technique disclosed in this specification.

With the use of the technique disclosed in this specification fordistribution of images of dangerous places, the following advantages (1)to (3) are obtained, for example.

(1) Quick Acquisition of Information on the Occurrence of Accident

To date, accident information has been broadcast by traffic informationservices. Also, a proposal has been made of a system for collectingimages by a plurality of viewpoints at the time of the occurrence of atraffic accident (for example, refer to Japanese Unexamined PatentApplication Publication No. 2012-98105). However, there is a delay fromthe occurrence of an accident to the arrival of information to eachdriver. Also, the target persons to whom the information is provided arelimited to vehicle drivers, and it is not possible for the pedestrians,and the like to get the information. In contrast, by the techniquedisclosed in this specification, it is possible to transmit imagescaptured not by in-vehicle cameras, but by viewpoints of various personswho encountered an accident or an incident, including pedestrians anddrivers, to pedestrians and drivers who are approaching the occurrencesite of the accident and the incident. Also it is possible to transmitinformation notifying a danger, such as images of an accident, anincident, and the like to those people in real time over amultifunctional information terminal, such as a head mounted display, asmart phone, or the like, or a head up display on a windshield. It is,therefore, possible to improve prevention efficiency of a seconddisaster and congestion.

(2) Prevention of Accidents

Even if a driver actually does not encounter an accident, there areminor incident operations having a high risk of causing an accident,such as an abrupt steering, a sudden braking, or the like (or predictionof the occurrence of an accident). Also, there are cases where althoughan accident has not actually occurred, or a pedestrian has not sufferedan injury, a minor incident behavior occurs, such as a pedestrianstumbles on an obstacle or a hollow on the road, and the like. With thetechnique disclosed in this specification, it is possible for the headmounted display worn by a driver or a pedestrian, or the like toautomatically detect a minor incident behavior on the basis ofbiological information, such as a pulse, perspiration, and the like, andtransmits a captured image of such a dangerous place to the informationmanagement center. Thereby, it is possible to share minor incidentinformation among the users in real time in order to prevent anaccident.

(3) Creation of Hazard Map

With the technique disclosed in this specification, images of thedangerous places that were obtained from a plurality of image capturingdevices are managed in database, and thus it is possible to create ahazard map of various accidents and incidents (injury, murder, theft,robbery, and the like) including traffic accidents, and to provide thehazard map to pedestrians and drivers in advance, or to give informationand guidance in connection with real-time traveling or walking points.Thereby, it is possible to prevent the occurrence of various accidentsand incidents. Also, it is possible to create and provide not only ahazard map of the accidents and incidents that actually happened, butalso a latent hazard map indicating failed accidents or incidents. Forexample, it is possible to collect information of a dangerous placewhere an accident is likely to occur, such as a place where a vehicledriver or a pedestrian encountered a minor incident from a head mounteddisplay worn by each driver or pedestrian.

With the technique disclosed in this specification, the captured imagesof a dangerous place and the accompanying information is collected inthe information management center, and is managed in an image database.The image database has various applications including the following (1)to (4) in addition to above-described applications.

(1) Cause analysis of an accident and establishment of futurecountermeasures from the standpoint of a road constructor.

(2) Guidance on amendment of Road Traffic Law and enforcement methodfrom the standpoints of Ministry of Land, Infrastructure, Transport andTourism, and the police.

(3) Expediting arrest of criminals from the standpoints of the police.

(4) Prospect of improvement of precision and speedup of lawsuit decisionsuch as indemnities, and gravity of punishments after an accident, andthe like from the standpoints of Civil Procedure Law and CriminalProcedure Law.

Also, in the above, a description has been mainly given of theembodiments in which an information distribution system according to thetechnique disclosed in this specification distributes informationregarding a dangerous place. However, it is thought that the system isoperated by replacing a “dangerous place” with a “dangerous thing”, suchas a dangerous person, a dangerous animal, or the like. The dangerousperson mentioned here includes an important dangerous person, such as awanted criminal, and a fugitive, and also includes a relativelyinsignificant dangerous person who make a nuisance, such as a violentperson, a smoker while walking, and the like. The dangerous animalincludes a biting pet, and a highly poisonous animal. In addition, thedangerous thing includes flammable materials that are easy to catchfire, to explode, and the like. It is assumed that a dangerous thingchanges positional information every moment unlike a dangerous place.Accordingly, the information management center may manage the imagedatabase in association with identification information based on arecognition result of the captured image, and the like in place ofperiodically updating positional information of the dangerous thing, ormanaging with the positional information.

In this regard, the image capturing device that provides an image of adangerous place is basically an information device that is used by beingworn by a user as a head mounted display. In the case of using a headmounted display, both a transmissive type and an immersion type may beused.

Also, the information display device on which information of a capturedimage of a dangerous place, and the like is displayed, which isdistributed from the information management center, is basically movedwith a user, but may be a smart phone, or a head up display in additionto a head mounted display.

In short, a description has been given of the technique disclosed inthis specification by exemplification. However, the description of thisspecification should not be construed in a limited manner. In order todetermine the gist of the technique disclosed in this specification, theappended claims should be considered.

In this regard, it is possible to configure the technique disclosed inthis specification as follows.

(1) An image capturing device including:

a display section attached to a head or a face of a user, and configuredto display an image;

a communication section configured to communicate with an externaldevice;

an image capturing section;

a positional information acquisition section configured to acquirecurrent positional information of the user; and

a control section configured to capture an image of a dangerous place bythe image capturing section in accordance with determination of adangerous state, and transmit a captured image from the communicationsection to the external device together with the positional informationacquired by the positional information acquisition section.

(2) The image capturing device according to (1), further including astate information acquisition section configured to acquire stateinformation of the user,

wherein the control section determines the dangerous state on the basisof the state information.

(3) The image capturing device according to (1), further including anenvironmental information acquisition section configured to acquireenvironmental information,

wherein the control section determines the dangerous state on the basisof the environmental information.

(4) The image capturing device according to (1),

wherein the control section transmits the captured image withinformation on image captured time.

(5) The image capturing device according to (1),

wherein the control section transmits the captured image withinformation on attribute information of the user.

(6) The image capturing device according to (2),

wherein the state information acquisition section acquires biologicalinformation of the user, and

the control section determines the dangerous state by itself on thebasis of the biological information, or acquires a determination resultof a relative dangerous state determined by an external apparatusrecording a large number of pieces of data.

(7) The image capturing device according to (6),

wherein the control section determines the dangerous state on the basisof at least any one piece of biological information among a pulse of theuser, perspiration, myoelectric potential, and movement of an eyeball.

(8) The image capturing device according to (6),

wherein the control section determines the dangerous state further inconsideration of whether the user is viewing a content using the displaysection.

(9) The image capturing device according to (8), further including anenvironmental information acquisition section configured to acquireenvironmental information,

wherein the control section determines the dangerous state inconsideration of the environmental information.

(10) The image capturing device according to (3) or (9),

wherein the control section determines the dangerous state on the basisof an analysis result of ambient noises, winds, atmosphere (a thick fogor smoke, radioactive rays, or toxic chemical substance in anatmosphere), surrounding captured images as the environmentalinformation.

(11) The image capturing device according to (6), further including aline of sight acquisition section configured to acquire a line of sightof the user,

wherein the control section determines the dangerous state on the basisof biological information of the user when the line of sight falls on aplace predicted to be dangerous.

(12) The image capturing device according to (11), further including anenvironmental information acquisition section configured to acquireenvironmental information,

wherein the control section identifies the place predicted to bedangerous on the basis of the environmental information.

(13) The image capturing device according to (1), further including aline of sight acquisition section configured to acquire a line of sightof the user,

wherein when the control section determines the dangerous state, thecontrol section captures a still image or a moving image by the imagecapturing section in accordance with whether or not the line of sightremained for a certain time period.

(14) The image capturing device according to (13),

wherein the control section stops capturing the image in accordance witha change in the line of sight while capturing the moving image.

(15) The image capturing device according to (1), further including aline of sight acquisition section configured to acquire a line of sightof the user,

wherein the control section identifies the dangerous place, startscapturing a moving image in accordance with the line of sight falling onthe dangerous place, and stops capturing the moving image in accordancewith the line of sight falling off the dangerous place.

(16) The image capturing device according to (1),

wherein the control section determines whether to capture a still imageor a moving image in accordance with a level of importance of thedangerous place or a type of the dangerous place.

(17) A method of acquiring information, including:

determining a dangerous state on the basis of state information orenvironmental information of a user;

acquiring positional information in order to acquire current positionalinformation of the user;

capturing an image of a dangerous place by an image capturing section inaccordance with determination of the dangerous state; and

transmitting a captured image from a communication section to anexternal device together with the positional information acquired by theacquiring positional information.

(18) An information distribution system including:

an image database configured to manage an image of a dangerous placetogether with positional information; and

an information providing section configured to retrieve an image of adangerous place corresponding to a current position of an informationdisplay device from the image database in order to provide the image.

(19) The information distribution system according to (18),

wherein the images of the dangerous place collected from one or moreimage capturing devices are managed with individual pieces of positionalinformation, respectively, in the image database.

(20) The information distribution system according to (19),

wherein time information when the image of the dangerous place wascaptured by the image capturing device is managed in the image databasein combination, and

the information providing section retrieves an image of the dangerousplace corresponding to the current position of the information displaydevice from the image database by filtering with a current time.

(21) The information distribution system according to (19),

further including a user database configured to manage attributeinformation of a user of each image capturing device,

wherein the information providing section retrieves an image of thedangerous place corresponding to the current position of the informationdisplay device from the image database by filtering with attributeinformation of a user of an information display device of the providingdestination in order to provide the image.

(22) The information distribution system according to (18),

wherein the information providing section provides a hazard map createdby mapping each dangerous place stored in the image database onto a map.

(23) The information distribution system according to (22),

wherein the information providing section creates a hazard map for eachdanger category.

(24) The information distribution system according to (22),

wherein the information providing section creates a hazard map for eachuser attribute.

(25) The information distribution system according to (18),

wherein when a plurality of images are retrieved for the currentposition of the information display device, the information providingsection narrows down the images to a predetermined number of imagesusing any one of the following methods,

(A) performing an image search using a default search condition,(B) performing an image search using a search condition specified by aninformation display device, and(C) transmitting necessary information to an information display device,and the information display device performing an image search using adesired search condition.

What is claimed is:
 1. An image capturing device, comprising: a displaysection configured to be attached to a head or a face of a user; acommunication section configured to communicate with an external device;an image capturing section; a positional information acquisition sectionconfigured to acquire positional information of the user; a controlsection configured to control the image capturing section, to capture animage of a place, based on a state of the user and control thecommunication section to transmit the captured image and the acquiredpositional information to the external device.
 2. The image capturingdevice according to claim 1, further comprising: a state informationacquisition section configured to acquire biological information of theuser, wherein the control section is configured to control the imagecapturing section, to capture the image of the place, based on the stateof the user and the acquired biological information.
 3. The imagecapturing device according to claim 1, further comprising: a stateinformation acquisition section configured to acquire state informationof the user; and an environmental information acquisition sectionconfigured to acquire environmental information, wherein the controlsection is configured to determine the state based on the acquired stateinformation or the acquired environmental information.
 4. The imagecapturing device according to claim 1, wherein the control section isconfigured to transmit the captured image with at least one ofinformation on image captured time or attribute information of the user.5. The image capturing device according to claim 1, wherein the controlsection is configured to acquire a determination result of a relativestate determined by an external apparatus which is configured to recorda number of pieces of data.
 6. The image capturing device according toclaim 2, wherein the control section is configured to determine thestate based on at least one piece of the biological information among apulse of the user, perspiration, myoelectric potential, or movement ofan eyeball.
 7. The image capturing device according to claim 1, whereinthe control section is configured to determine the state based on adetermination that the user views content on the display section.
 8. Theimage capturing device according to claim 3, wherein the control sectionis configured to determine the state on the basis of an analysis resultof ambient noises, winds, or atmosphere (a thick fog or smoke,radioactive rays, or toxic chemical substance in an atmosphere) as theenvironmental information.
 9. The image capturing device according toclaim 2, further comprising: a line of sight acquisition sectionconfigured to acquire a line of sight of the user, wherein the controlsection is configured to determine the state based on the biologicalinformation of the user and a prediction of the place to be dangerouswhere the line of sight falls.
 10. The image capturing device accordingto claim 9, wherein the control section is configured to identify theplace predicted to be dangerous based on environmental information. 11.The image capturing device according to claim 1, further comprising: aline of sight acquisition section configured to acquire a line of sightof the user, wherein the control section is configured to capture astill image or a moving image by the image capturing section based onthe state and a determination that the line of sight remained for acertain time period.
 12. The image capturing device according to claim11, wherein the control section is configured to stop the capture of theimage based on a change in the line of sight.
 13. The image capturingdevice according to claim 1, further comprising: a line of sightacquisition section configured to acquire a line of sight of the user,wherein the control section is configured to identify the place, startthe capture of a moving image based on the line of sight which falls onthe place, and stop the capture of the moving image based on the line ofsight which falls off the place.
 14. The image capturing deviceaccording to claim 1, wherein the control section is configured todetermine whether to capture a still image or a moving image based on alevel of importance of the place or a type of the place.
 15. A method ofacquiring information, comprising: determining a state of a user basedon state information of the user; acquiring positional information ofthe user; capturing an image of a place based on the determined state;and transmitting the captured image and the acquired positionalinformation to an external device.
 16. A non-transitorycomputer-readable medium having stored thereon, computer-executableinstructions for causing a computer to perform operations, comprising:determining a state of a user based on state information of the user;acquiring positional information of the user; capturing an image of aplace based on the determined state; and transmitting the captured imageand the acquired positional information to an external device.